Evaluation of Loop Scheduling Algorithms on DistributedMemory Systems

Loops are the largest source of parallelism in many applications. All prior DOALL loop scheduling algorithms such as Self-Scheduling, Guided Self-Scheduling, Trapezoid Self-Scheduling, and Factoring try to achieve workload balance through decreasing chunk sizes. Moreover, they have been analyzed only for shared memory platforms. In this work, the prior loop scheduling methods will be evaluated on two distributed memory machines using realistic workloads from the NAS Parallel benchmark suite and Livermore Loop Series. The distributed memory platforms are: a 16-node IBM SP2 and a 16-node nCUBE 2. The experimental results show that these decreasing chunk size methods tend to increase the communication time in distributed memory models by assigning more chunks. In view of these results, two new schemes, called Fixed Increase and Variable Increase, are introduced. Contrary to the earlier techniques, these schemes increase the chunk sizes in order to minimize the scheduling overhead by reducing interpro-cessor communication. The new algorithms can be implemented by parallel compilers and are scalable over large numbers of processors and iterations. Extensive measurements on both the machines indicate that the increasing chunk size methods can provide better performance than the existing algorithms for almost all workload patterns.